Task-Oriented Synergistic Multimodality

Multimodal interface aims at overcoming the shortcomings of conventional human-computer communication -serialized input, excessive accuracy, and low input efficiency -by exploiting the complementary nature of multiple modalities in interpreting user intent, thus improving the efficiency and naturalness of humancomputer interaction. In this paper, we present motivations driving multimodal interface research and give a brief introduction to the ideas underlying our MUIR* project. We put forward a task-oriented framework of multimodal interface research, which consists of our concept of multimodality and perspectives concerning sufficiency vs. accuracy, multimodal integration approach, and an agent-based architecture for multimodal interface.